Image processing method and image processing device

ABSTRACT

An image processing method and an image processing device are provided. The image processing method includes: determining a first width and a second width, where the second width is the width of a gap between display devices of target images on N screens, the first width is the width of a blind spot between source images corresponding to the N screens, N is an integer greater than 1, and the N screens are of a same size and are arranged side by side at a same height; and when the first width is different from the second width, adjusting the source images according to the determined first width and second width so as to obtain the target images, so that no mismatch exists in the stitched target images on the N screens.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Patent ApplicationNo. PCT/CN2012/081207, filed on Sep. 10, 2012, which is herebyincorporated by reference in its entirety.

TECHNICAL FIELD

Embodiments of the present invention relate to the field of videoconferencing, and in particular, to an image processing method and animage processing device.

BACKGROUND

As coding and information compression technologies develop and digitalnetworks rocket, videoconferencing systems start to step into themarket. When IP (Internet Protocol, Internet Protocol) networktechnologies move toward maturity, IP networks become an importantnetwork platform for global communications. The ITU-T (InternationalTelecommunication Union-Telecommunication Standardization Sector,International Telecommunication Union-Telecommunication StandardizationSector) puts forward the H.323 videoconferencing standard based on IPnetworks. This swerves the research and application direction of avideoconferencing system to development based on IP networks.

Today, a videoconferencing system based on the H.323 videoconferencingstandard is finding increasingly wide use with the rapid development ofIP networks. Most of departments, such as the government, the army, andenterprises, have deployed their own videoconferencing system to improvemeeting efficiency and save meeting costs.

Telepresence (Telepresence), a new technology, emerges in thevideoconferencing market. Remote participants are at another end of anetwork, but their images are displayed in true-to-life dimensions on adisplay device in a conference room, and a user does not feel anydiscomfort or difference. A perfect combination of a video image intrue-to-life dimensions, a high-definition image effect, an audio systemwith a sense of space, and the environment creates a sense of being inthe same conference room with the remote participants. A complete set ofa telepresence system needs to consist of an appropriate conferenceroom, power, a high-bandwidth network channel, an independent wall, adesk, a chair, a light, a video display device, a control device, and anaudio device. Compared with a conventional conferencing system, atelepresence system has its unique advantages, that is, it not onlysaves travel expenses but also significantly improves conferencequality. In addition, telepresence overcomes the plane and unrealeffects of a conventional video conference and makes a conference morenatural, thereby improving conference efficiency.

In the telepresence technology, each camera at a local conference sitecorresponds to a different user area, and each camera simultaneouslycaptures an image of a corresponding user area and sends it to aconferencing terminal at a remote conference site; the conferencingterminal at the remote conference site stitches, by using a physical ordigital image stitching technology, images captured by cameras at thelocal conference site, and then outputs them to adjacent display devicesat the remote conference site for displaying; and a conferencingterminal at the local conference site may also stitch, by using aphysical or digital image stitching technology, images captured bycameras at the remote conference site, and then outputs them to adjacentdisplay devices at the local conference site for displaying. For thepurpose of implementing image stitching without a mismatch, a certainfixed model of display devices are generally selected at the localconference site and the remote conference site, or at least frames ofthe display devices are the same.

In the prior art, image stitching without a mismatch requires that thewidth of a blind spot between two adjacent images equal the minimum edgegap between displayable areas of two adjacent display devices. Theminimum edge gap may also be called the width of a gap between displaydevices. If the displayable areas of the display devices occupy theentire screens of the display devices, the width of a gap between thedisplay devices basically equals the sum of the width of frames of thetwo adjacent display devices. Such stitching without a mismatch will notmake a user feel that an image is mismatched, thereby better satisfyinga reality requirement of the telepresence technology. Here, the blindspot is a conference scenario area that is not covered by the adjacentimages. If the adjacent images are images shot by adjacent cameras, theblind spot is specifically a conference scenario area that is beyondshooting areas of the adjacent cameras. A displayable area is thelargest display area that hardware of a display device can support, andthe area is generally rectangular.

For the purpose of implementing image stitching without a mismatch, itis required that the width of a blind spot between adjacent images equalthe width of a gap between adjacent display devices that are used todisplay the adjacent images at a target end. However, as display devicesare continually replaced by new ones, screen dimensions and frame sizesof the display devices change accordingly. If the width of a gap betweendisplay devices at a target end is inconsistent with the width of ablind spot between the adjacent images, this may lead to a mismatch inimage stitching at the target end, thereby affecting user experience.

SUMMARY

Embodiments of the present invention provide an image processing methodand an image processing device, so as to improve a sense of reality oftelepresence.

In a first aspect, an image processing method is provided, including:determining a first width and a second width, where the second width isthe width of a gap between display devices of target images on Nscreens, the first width is the width of a blind spot between sourceimages corresponding to the N screens, N is an integer greater than 1,and the N screens are of a same size and are arranged side by side at asame height; and when the first width is different from the secondwidth, adjusting the source images according to the determined firstwidth and second width so as to obtain the target images, so that nomismatch exists in the stitched target images on the N screens.

With reference to the first aspect, in an implementation manner of thefirst aspect, the adjusting the source images so as to obtain the targetimages, so that no mismatch exists in the stitched target images on theN screens includes: pruning and padding a source image corresponding toone or more screens among the N screens so as to obtain a correspondingtarget image, so that no mismatch exists in the stitched target imageson the N screens; or zooming out and padding a source imagecorresponding to one or more screens among the N screens so as to obtaina corresponding target image, so that no mismatch exists in the stitchedtarget images on the N screens; or zooming in and pruning a source imagecorresponding to one or more screens among the N screens so as to obtaina corresponding target image, so that no mismatch exists in the stitchedtarget images on the N screens.

With reference to the first aspect and the preceding implementationmanner, in another implementation manner of the first aspect, thepruning and padding a source image corresponding to one or more screensamong the N screens so as to obtain a corresponding target image, sothat no mismatch exists in the stitched target images on the N screensincludes: calculating a part that needs to be pruned from the sourceimage, determining, in a color array of the source image, a pixel areacorresponding to the part that needs to be pruned, and discarding pixeldata in the pixel area; and calculating a part that needs to be padded,determining, in the color array of the source image, a position of apixel area corresponding to the part that needs to be padded, and addingpixel data of a preset color into the pixel area corresponding to theposition of the pixel area, so as to obtain the target imagecorresponding to the source image, so that no mismatch exists in thestitched target images on the N screens.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, thezooming out and padding a source image corresponding to one or morescreens among the N screens so as to obtain a corresponding targetimage, so that no mismatch exists in the stitched target images on the Nscreens includes: calculating a zoom factor according to the first widthand the second width; and when the first width is greater than thesecond width, compressing all pixels in a color array of the sourceimage according to the zoom factor, calculating a part that needs to bepadded, determining, in the color array of the source image, a positionof a pixel area corresponding to the part that needs to be padded, andadding pixel data of a preset color into the pixel area corresponding tothe position of the pixel area, thereby obtaining the target imagecorresponding to the source image and avoiding a mismatch in thestitched target images on the N screens.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, thezooming in and pruning a source image corresponding to one or morescreens among the N screens so as to obtain a corresponding targetimage, so that no mismatch exists in the stitched target images on the Nscreens includes:

calculating a zoom factor according to the first width and the secondwidth; and when the first width is less than the second width,stretching all pixels in a color array of the source image according tothe zoom factor, calculating a part that needs to be pruned from thesource image, determining, in the color array of the source image, apixel area corresponding to the part that needs to be pruned, anddiscarding pixel data in the pixel area, thereby obtaining the targetimage corresponding to the source image and avoiding a mismatch in thestitched target images on the N screens.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, thepruning and padding a source image corresponding to one or more screensamong the N screens so as to obtain a corresponding target image, sothat no mismatch exists in the stitched target images on the N screensincludes: taking a source image corresponding to a first screen amongthe N screens as a target image on the first screen, or pruning andpadding a source image corresponding to a first screen as a target imageon the first screen; and pruning and padding a source imagecorresponding to an adjacent screen of the first screen as a targetimage on the adjacent screen, so that no mismatch exists in the stitchedtarget images on the N screens.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, thepruning and padding a source image corresponding to an adjacent screenof the first screen as a target image on the adjacent screen includes:when the first width is greater than the second width, pruning a partthat has a width of c from a side that is in the source imagecorresponding to the adjacent screen and is away from the first screen,and padding an edge that is in the preset color and has a width of cinto a side that is in the source image corresponding to the adjacentscreen and is near the first screen, so as to obtain the target image onthe adjacent screen, where c=a−b, a is the first width, and b is thesecond width.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, thepruning and padding a source image corresponding to an adjacent screenof the first screen as a target image on the adjacent screen includes:when the first width is less than the second width, pruning a part thathas a width of c from a side that is in the source image correspondingto the adjacent screen and is near the first screen, and padding an edgethat is in the preset color and has a width of c into a side that is inthe source image corresponding to the adjacent screen and is away fromthe first screen, so as to obtain the target image on the adjacentscreen, where c=b−a, a is the first width, and b is the second width.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, thepruning a source image corresponding to a first screen as a target imageon the first screen includes: pruning an edge that has a width of f fromtwo sides of the source image corresponding to the first screen, andseparately padding an edge that is in the preset color and has a widthoff into the two sides of the source image corresponding to the firstscreen, where f<=|a−b|.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, f=|a−b|/2or f=|a−b|, a is the first width, and b is the second width.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, thezooming out and padding a source image corresponding to one or morescreens among the N screens so as to obtain a corresponding targetimage, so that no mismatch exists in the stitched target images on the Nscreens includes: when the first width is greater than the second width,zooming out the source image according to a zoom factor k, wherek=(b+d)/(a+d), a is the first width, b is the second width, and d is thewidth of the source image; and padding an edge in the preset coloraround the source image after zooming out, so as to obtain the targetimage.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, thezooming in and pruning a source image corresponding to one or morescreens among the N screens so as to obtain a corresponding targetimage, so that no mismatch exists in the stitched target images on the Nscreens includes: when the first width is less than the second width,zooming in the source image according to a zoom factor k, wherek=(b+d)/(a+d), a is the first width, b is the second width, and d is thewidth of the source image; and pruning an edge part from the sourceimage after zooming in, so as to obtain the target image.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, thepreset color is a frame color of a display device of the target image.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, themethod further includes: preprocessing original images corresponding tothe N screens so as to obtain source images that correspond to the Nscreens and have same dimensions as the target images, where dimensionsof the original images are the same as or different from dimensions ofthe target images.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, thepreprocessing original images corresponding to the N screens so as toobtain source images that correspond to the N screens and have samedimensions as the target images includes: if the dimensions of theoriginal images are the same as the dimensions of the target images,taking the original images as the source images; and if the dimensionsof the original images are different from the dimensions of the targetimages, separately zooming in or out, pruning, and/or padding theoriginal images corresponding to the N screens so as to obtaincorresponding source images.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, the ifthe dimensions of the original images are different from the dimensionsof the target images, separately zooming in or out, pruning, and/orpadding the original images corresponding to the N screens so as toobtain corresponding source images includes: if aspect ratios of theoriginal images are the same as aspect ratios of the target images andthe width w of the original images is different from the width d of thetarget images, proportionally zooming in or out, according to a zoomfactor m, the original images corresponding to the N screens so as toobtain the corresponding source images, where m=d/w.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, thedetermining a first width includes: determining that the first widtha=n×A, where n=m, and A is the width of a blind spot between theoriginal images corresponding to the N screens.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, the ifthe dimensions of the original images are different from the dimensionsof the target images, separately zooming in or out, pruning, and/orpadding the original images corresponding to the N screens so as toobtain corresponding source images includes: if aspect ratios of theoriginal images are different from aspect ratios of the target images,proportionally zooming in or out, according to a zoom factor m1, theoriginal images corresponding to the N screens so as to obtain thecorresponding intermediate images, where m1=e/h, e is the height of thetarget images, and h is the height of the original images; and pruningor padding an edge that has a total width of |d−(ew/h)| from or into oneor two sides of the intermediate images corresponding to the N screens,so as to obtain the corresponding source images, where d is the width ofthe target images, w is the width of the original images, and ∥ is acalculation of taking an absolute value.

With reference to the first aspect and the preceding implementationmanners, in another implementation manner of the first aspect, thedetermining a first width includes: determining that the first widtha=n×A, where n=(2Ae+ew−dh)/2Ah, and A is the width of a blind spotbetween the original images corresponding to the N screens.

In a second aspect, an image processing device is provided, including: adetermining unit, configured to determine a first width and a secondwidth, where the second width is the width of a gap between displaydevices of target images on N screens, the first width is the width of ablind spot between source images corresponding to the N screens, N is aninteger greater than 1, and the N screens are of a same size and arearranged side by side at a same height; and an adjusting unit,configured to adjust, when the first width is different from the secondwidth, the source images according to the first width and the secondwidth that are determined by the determining unit, so as to obtain thetarget images, so that no mismatch exists in the stitched target imageson the N screens.

With reference to the second aspect, in an implementation manner of thesecond aspect, the adjusting unit is specifically configured to: pruneand pad a source image corresponding to one or more screens among the Nscreens so as to obtain a corresponding target image, so that nomismatch exists in the stitched target images on the N screens; or zoomout and pad a source image corresponding to one or more screens amongthe N screens so as to obtain a corresponding target image, so that nomismatch exists in the stitched target images on the N screens; or zoomin and prune a source image corresponding to one or more screens amongthe N screens so as to obtain a corresponding target image, so that nomismatch exists in the stitched target images on the N screens.

With reference to the second aspect and the preceding implementationmanner, in another implementation manner of the second aspect, theadjusting unit is specifically configured to: calculate a part thatneeds to be pruned from the source image, determine, in a color array ofthe source image, a pixel area corresponding to the part that needs tobe pruned, and discard pixel data in the pixel area; and calculate apart that needs to be padded, determine, in the color array of thesource image, a position of a pixel area corresponding to the part thatneeds to be padded, and add pixel data of a preset color into the pixelarea corresponding to the position of the pixel area, thereby obtainingthe target image corresponding to the source image and avoiding amismatch in the stitched target images on the N screens; or, theadjusting unit is specifically configured to: calculate a zoom factoraccording to the first width and the second width; and when the zoomfactor is less than 1, compress all pixels in a color array of thesource image according to the zoom factor, calculate a part that needsto be padded, determine, in the color array of the source image, aposition of a pixel area corresponding to the part that needs to bepadded, and add pixel data of a preset color into the pixel areacorresponding to the position of the pixel area, thereby obtaining thetarget image corresponding to the source image and avoiding a mismatchin the stitched target images on the N screens; or, the adjusting unitis specifically configured to: calculate a zoom factor according to thefirst width and the second width; and when the zoom factor is greaterthan 1, stretch all pixels in a color array of the source imageaccording to the zoom factor, calculate a part that needs to be prunedfrom the source image, determine, in the color array of the sourceimage, a pixel area corresponding to the part that needs to be pruned,and discard pixel data in the pixel area, thereby obtaining the targetimage corresponding to the source image and avoiding a mismatch in thestitched target images on the N screens.

With reference to the second aspect and the preceding implementationmanners, in another implementation manner of the second aspect, theadjusting unit is specifically configured to take a source imagecorresponding to a first screen among the N screens as a target image onthe first screen, or prune and pad a source image corresponding to afirst screen as a target image on the first screen; and prune and pad asource image corresponding to an adjacent screen of the first screen asa target image on the adjacent screen, so that no mismatch exists in thestitched target images on the N screens.

With reference to the second aspect and the preceding implementationmanners, in another implementation manner of the second aspect, theadjusting unit is specifically configured to: when the first width isgreater than the second width, prune a part that has a width of c from aside that is in the source image corresponding to the adjacent screenand is away from the first screen, and pad an edge that is in the presetcolor and has a width of c into a side that is in the source imagecorresponding to the adjacent screen and is near the first screen, so asto obtain the target image on the adjacent screen, where c=a−b, a is thefirst width, and b is the second width; or

the adjusting unit is specifically configured to: when the first widthis less than the second width, prune a part that has a width of c from aside that is in the source image corresponding to the adjacent screenand is near the first screen, and pad an edge that is in the presetcolor and has a width of c into a side that is in the source imagecorresponding to the adjacent screen and is away from the first screen,so as to obtain the target image on the adjacent screen, where c=b−a, ais the first width, and b is the second width; or

the adjusting unit is specifically configured to prune an edge that hasa width of f from two sides of the source image corresponding to thefirst screen, and separately pad an edge that is in the preset color andhas a width of f into the two sides of the source image corresponding tothe first screen, where f<=|a−b|.

With reference to the second aspect and the preceding implementationmanners, in another implementation manner of the second aspect, theadjusting unit is specifically configured to: when the first width isgreater than the second width, zoom out the source image according to azoom factor k, where k=(b+d)/(a+d), a is the first width, b is thesecond width, and d is the width of the source image, and pad an edge inthe preset color around the source image after zooming out, so as toobtain the target image; or, the adjusting unit is specificallyconfigured to: when the first width is less than the second width, zoomin the source image according to a zoom factor k, where k=(b+d)/(a+d), ais the first width, b is the second width, and d is the width of thesource image, and prune an edge part from the source image after zoomingin, so as to obtain the target image.

With reference to the second aspect and the preceding implementationmanners, in another implementation manner of the second aspect, theadjusting unit is further configured to preprocess original imagescorresponding to the N screens so as to obtain source images thatcorrespond to the N screens and have same dimensions as the targetimages, where dimensions of the original images are the same as ordifferent from dimensions of the target images.

With reference to the second aspect and the preceding implementationmanners, in another implementation manner of the second aspect, theadjusting unit is specifically configured to: if the dimensions of theoriginal images are the same as the dimensions of the target images,take the original images as the source images; and if the dimensions ofthe original images are different from the dimensions of the targetimages, separately zoom in or out, prune, and/or pad the original imagescorresponding to the N screens so as to obtain corresponding sourceimages.

With reference to the second aspect and the preceding implementationmanners, in another implementation manner of the second aspect, theadjusting unit is specifically configured to: if aspect ratios of theoriginal images are the same as aspect ratios of the target images andthe width w of the original images is different from the width d of thetarget images, proportionally zoom in or out, according to a zoom factorm, the original images corresponding to the N screens so as to obtaincorresponding source images, where m=d/w; and the determining unit isspecifically configured to determine that the first width a=n×A, wheren=m, and A is the width of a blind spot between the original imagescorresponding to the N screens.

With reference to the second aspect and the preceding implementationmanners, in another implementation manner of the second aspect, theadjusting unit is specifically configured to: if aspect ratios of theoriginal images are different from aspect ratios of the target images,proportionally zoom in or out, according to a zoom factor m1, theoriginal images corresponding to the N screens so as to obtaincorresponding intermediate images, where m1=e/h, e is the height of thetarget images, and h is the height of the original images; and prune orpad an edge that has a total width of |d−(ew/h)| from or into one or twosides of the intermediate images corresponding to the N screens, so asto obtain corresponding source images, where d is the width of thetarget images, w is the width of the original images, and ∥ is acalculation of taking an absolute value; and the determining unit isspecifically configured to determine that the first width a=n×A, wheren=(2Ae+ew−dh)/2Ah, and A is the width of a blind spot between theoriginal images corresponding to the N screens.

According to the embodiments of the present invention, the width of ablind spot between source images and the width of a gap between displaydevices of target images are acquired, and the source images areadjusted according to the acquired width values so as to obtain stitchedtarget images without a mismatch, thereby improving a sense of realityof telepresence.

BRIEF DESCRIPTION OF DRAWINGS

To illustrate the technical solutions in the embodiments of the presentinvention more clearly, the following briefly introduces theaccompanying drawings required for describing the embodiments.Apparently, the accompanying drawings in the following description showmerely some embodiments of the present invention, and a person ofordinary skill in the art may still derive other drawings from theseaccompanying drawings without creative efforts.

FIG. 1A and FIG. 1B are schematic diagrams of scenarios to whichembodiments of the present invention are applicable;

FIG. 2 is a schematic flowchart of an image processing method accordingto an embodiment of the present invention;

FIG. 3A-FIG. 3C are schematic diagrams of examples of obtaining sourceimages from original images;

FIG. 4A is a schematic diagram of an image processing manner accordingto an embodiment of the present invention;

FIG. 4B is a schematic diagram of an image processing manner accordingto another embodiment of the present invention;

FIG. 5 is a schematic diagram of an image processing manner according toanother embodiment of the present invention;

FIG. 6 is a schematic diagram of an image processing manner according toanother embodiment of the present invention;

FIG. 7 is a schematic diagram of an image processing manner according toanother embodiment of the present invention;

FIG. 8 is a schematic diagram of an image processing manner according toanother embodiment of the present invention;

FIG. 9 is a schematic diagram of an image processing manner according toanother embodiment of the present invention;

FIG. 10 is a schematic diagram of an image processing manner accordingto another embodiment of the present invention;

FIG. 11 is a block diagram of an image processing device according to anembodiment of the present invention; and

FIG. 12 is a block diagram of an image processing device according toanother embodiment of the present invention.

DESCRIPTION OF EMBODIMENTS

The following clearly describes the technical solutions in theembodiments of the present invention with reference to the accompanyingdrawings in the embodiments of the present invention. Apparently, thedescribed embodiments are merely a part rather than all of theembodiments of the present invention. All other embodiments obtained bya person of ordinary skill in the art based on the embodiments of thepresent invention without creative efforts shall fall within theprotection scope of the present invention.

FIG. 1A and FIG. 1B are schematic diagrams of scenarios to whichembodiments of the present invention are applicable. In the followingembodiments, it is assumed that both source images and target imagesoccupy 3 screens. However, the embodiments of the present invention arenot limited to this. The number N of screens occupied by the sourceimages and that of screens occupied by the target images may be anyinteger greater than 1. In addition, it is assumed that the 3 screensare of a same size and are arranged side by side at a same height.

The arranging side by side may specifically be arranging side by side ina straight line and may also be arranging side by side in a curve or ina zigzag line.

The same height refers to that bottoms or tops of the N screens are at asame level.

The upper part of FIG. 1A is a schematic diagram showing source images101 a, 101 b, and 101 c. The schematic diagram illustrates three virtualdisplay devices 102 (shaded parts in FIG. 1A) of a same specification. Avirtual display device 102 is a display device that is expected todisplay a source image 101 when the source image 101 is generated, butmay not certainly conform to an actual specification of a display deviceat a target end, for example, the width, the height, and an aspect ratiomay be different to a certain extent. The width refers to a dimension ina horizontal direction, and the height refers to a dimension in avertical direction. If it is unnecessary to distinguish three sourceimages 101 a-101 c from each other, they are collectively called sourceimages 101 below. The source images 101 are generated according to thespecification of the display devices 102. For the purpose ofimplementing stitching without a mismatch, it is required that the widtha of a blind spot between the source images 101 equal the width of a gapbetween the display devices 102, that is, doubling the width of a leftframe or a right frame of a display device 102. As shown in FIG. 1A,stitching the source images 101 without a mismatch enables lines 109 a,109 b, and 109 c to be in a same straight line.

It should be noted that, in the description of the embodiments of thepresent invention, it is assumed that a displayable area of a displaydevice occupies the entire screen of the display device, that is, nonon-displayable area exists on the screen of the display deviceTherefore, the frame here may be a physical frame for adjoining twodisplay devices. However, the embodiments of the present invention donot set a limitation to this. For example, if there is a non-displayablearea inside the physical frame of a display device, the width of a gapbetween display devices needs also to include the corresponding width ofthe non-displayable area. This implementation manner also falls withinthe scope of the embodiments of the present invention.

The lower part of FIG. 1A is a schematic diagram of three displaydevices 103 of the target images, where the display devices 103 alsoseparately display the source images 101 a-101 c. Dimensions of adisplay area of a display device 103 are the same as those of a displayarea of a display device 102, but the width of a frame of a displaydevice 103 is narrower than that of a display device 102. Therefore, thewidth b of a gap between the display devices 103 is less than a. In theprior art, the width b of a gap between the display devices 103 is notconsidered during generation of the source images 101, and only thewidth a of a blind spot is considered. In this case, if the sourceimages 101 are displayed on the display devices 103 without processing,a gradient (for example, a gradient relative to a horizontal line) of astraight line determined by two actual scenario points that areseparately displayed on two source images on adjacent screens may bedifferent from a gradient of a straight line determined by the twoactual scenario points that are separately displayed on two targetimages on adjacent screens. Visually, as shown in FIG. 1A, the lines 109a, 109 b, and 109 c are not in the same straight line any longer. Forexample, FIG. 1A illustrates two actual scenario points P1 and P2 thatare respectively located at corresponding positions in the lines 109 aand 109 b. In the source images in the upper part, a gradient of astraight line determined by P1 and P2 is a gradient of the line 109 a.However, in the target images in the lower part, a gradient of astraight line (a dot-dashed line at the lower part of FIG. 1A)determined by P1 and P2 is different from the gradient of the line 109a. This leads to a mismatch in image stitching and affects a sense ofreality of telepresence.

FIG. 1B is a scenario where the width of a frame of a display device 103is greater than that of a display device 102, that is, b is greater thana. As shown in FIG. 1B, similarly, it leads to that the lines 109 a, 109b, and 109 c are not located in the same straight line any longer, andthe gradient of a straight line determined by P1 and P2 on source imagesis different from that determined by P1 and P2 on target images. Thisleads to a mismatch in image stitching and affects a sense of reality oftelepresence.

FIG. 2 is a schematic flowchart of an image processing method accordingto an embodiment of the present invention. The method in FIG. 2 may beexecuted by an image processing device at a target end. However, theembodiment of the present invention is not limited to this, and theimage processing device may also be located on other entities apart fromthe target end.

201. Determine a first width and a second width. The second width is thewidth of a gap between display devices of target images on N screens,for example, the width b shown in FIG. 1A-FIG. 3C. The first width isthe width of a blind spot between source images corresponding to the Nscreens, for example, the width a shown in FIG. 1A-FIG. 3C. N is aninteger greater than 1.

Here, the width b of a gap between the display devices of the targetimages may be the width of a gap between actual display devices at atarget conference site, for example, the width b approximately doublesthe width of frame of the actual display device. The width b of a gapmay also be the width of a gap between virtual display devices. Forexample, if a projection display manner is used at the target conferencesite, the width b of a gap may be an edge distance between projectionareas of a projection curtain that are scheduled to display targetimages on two adjacent screens. For another example, if an image to bedisplayed is proportionally zoomed in or out (for example, by adjustinga distance between a projector and a projection curtain or through imageprocessing) at the target conference site, the width b of a gap may bethe width of a gap obtained after the corresponding proportionalzooming. These changes all fall within the scope of the embodiment ofthe present invention.

The source images may specifically be images (referred to as originalimages below) shot by a camera at a source conference site and may alsobe images obtained by preprocessing the original images.

Specifically, if the dimensions of the original images are the same asthose of the target images, that is, the same as dimensions ofdisplayable areas of the display devices of the target images, forexample, both the width and the height are the same, the original imagesare taken as the source images, and the width of a blind spot betweenthe original images is taken as the width of a blind spot between thesource images. If the dimensions of the original images are differentfrom those of the target images, that is, different from dimensions ofdisplay areas of the display devices of the target images, for example,the width, the height, or aspect ratios are different, the originalimages are zoomed in or out, pruned, and/or padded to obtain the sourceimages. In this case, Width of a blind spot between the sourceimages=Width of a blind spot between the original images×Correspondingzoom factor (for example, the zoom factor n shown in FIG. 1C-FIG. 3C).

The following describes exemplary implementation manners for determiningthe width of a blind spot between the original images.

The width of a blind spot between the original images may specificallybe determined by an image processing device according to a shootingparameter (for example, coordinates of a shooting area of a camera) of acamera for shooting a source image, where the shooting parameter is sentby the source conference site; may also be determined by an imageprocessing device according to a specification parameter (for example,the width of a gap between display devices at the source conferencesite) of display devices at the source conference site, where thespecification parameter is sent by the source conference site; and maystill be directly sent by the source conference site.

The width of a blind spot between the original images may be calculatedaccording to coordinates of shooting areas of adjacent cameras. Forexample, the horizontal-coordinate values of left and right edges of ashooting area of a first camera of the two adjacent cameras are 100 mand 200 m respectively, and the horizontal-coordinate values of left andright edges of a shooting area of the other camera are 210 m and 310 mrespectively, and therefore, an area between the horizontal coordinate200 m and the horizontal coordinate 210 m is beyond the shooting area,and this area is a blind spot, that is, a blind spot between adjacentimages shot by the adjacent cameras, and the width of the blind spot isthe difference between 200 m and 210 m, that is, 10 m. The width of ablind spot between the original images may specifically be determinedthrough calculation according to coordinates of a shooting area that aresent by the source conference site, and may also be directly receivedfrom the source conference site after being calculated by the sourceconference site according to coordinates of a shooting area.

If an image shot by a camera is displayed in advance on a display deviceat a conference site (that is, the source conference site) where thecamera is located and a shooting angle of a shooting device is adjustedso that the shot image is stitched and displayed on the display devicewithout a mismatch, the width of a blind spot between the source imagesequals the width of a gap between display devices. The determining thewidth of a gap between the original images may specifically be receivingthe width of a gap between the display devices at the source conferencesite, where the width of a gap is sent by a device at the sourceconference site; and determining the received width as the width of ablind spot between the original images.

Optionally, a manner for sending a shooting parameter (for example,coordinates of a shooting area of a camera) of a camera, a specificationparameter of display devices (for example, the width of a gap betweendisplay devices at the source conference site), or the width of a blindspot may specifically be carried and sent through an extended media nameand transport address (media name and transport address) field (that isan “m=” line) of Session Description Protocol (SDP, Session DescriptionProtocol) information, that is, by carrying it into the media name andtransport address field during conference capability negotiation.

Optionally, as an embodiment, the second width may be extracted by animage processing device according to a system specification parameter(for example, a specification parameter of a display device) and mayalso be obtained according to an input of a user. The embodiment of thepresent invention does not set a limitation to this.

202. When the first width a is different from the second width b (a≠b),adjust the source images according to the determined first width andsecond width to obtain the target images, so that no mismatch exists inthe stitched target images on the N screens.

For the purpose of implementing stitching of target images without amismatch, it is required to ensure that a gradient (for example, agradient relative to a horizontal line) of a straight line determined bytwo actual scenario points that are separately displayed on two sourceimages on adjacent screens is identical to a gradient of a straight linedetermined by the two actual scenario points that are separatelydisplayed on two target images on adjacent screens. Visually, it isrequired that, in the target images on the N screens at the lower partsof FIG. 1A and FIG. 1B, lines 109 a, 109 b, and 109 c can still be in asame straight line. A display result in this case is identical to thatin an actual scenario, and this produces a better sense of reality.

According to the embodiment of the present invention, the width of ablind spot between source images and the width of a gap between displaydevices of target images are acquired, and the source images areadjusted according to the acquired width values so as to obtain stitchedtarget images without a mismatch, thereby improving a sense of realityof telepresence.

Optionally, as an embodiment, a manner for adjusting the source imagesto obtain the target images includes but is not limited to pruning,padding, zooming out, or zooming in a source image corresponding to oneor more screens among the N screens.

Specifically, an input interface of a monitor includes a digitalinterface and an analog interface. The digital interface includes a DVI(Digital Visual Interface, digital visual interface), an HDMI (HighDefinition Multimedia Interface, high definition multimedia interface),or the like, and the analog interface includes a VGA (Video GraphicsArray, video graphics array), or the like, where an input of the digitalinterface is an RGB color array, and an output of the analog interfaceis a YPbPr color array.

The pruning a source image may specifically be: calculating a part thatneeds to be pruned from the source image; determining, in a color arrayof the source image, a pixel area corresponding to the part that needsto be pruned; and discarding pixel data in the pixel area. The padding asource image may specifically be: calculating a part that needs to bepadded; determining, in the color array of the source image, a positionof a pixel area corresponding to the part that needs to be padded; andadding pixel data of a preset color into the pixel area corresponding tothe position of the pixel area, where the pixel data of the preset colormay specifically be RGB (125, 125, 125). The zooming out a source imagemay specifically be: calculating a zoom factor according to the firstwidth and the second width; and when the first width is greater than thesecond width, compressing all pixels in the color array of the sourceimage according to the zoom factor, where the compression mayspecifically be deleting pixel data of a part of pixel points accordingto the zoom factor. The zooming in a source image may specifically be:calculating a zoom factor according to the first width and the secondwidth; and when the first width is less than the second width,stretching all pixels in the color array of the source image accordingto the zoom factor, where the stretching may specifically be addingpixel data of a transitive color between adjacent pixel data accordingto the zoom factor, for example, adding, between two adjacent pixelpoints with pixel values of RGB (11, 11, 11) and RGB (33, 33, 33), a newpixel point with pixel data of RGB (22, 22, 22).

Optionally, as another embodiment, when a source image corresponding toone or more screens among the N screens is pruned and padded to obtain atarget image, a source image corresponding to a first screen among the Nscreens may be taken as a target image on the first screen, or a sourceimage corresponding to a first screen may be pruned and padded as atarget image on the first screen; and a source image corresponding to anadjacent screen of a first screen is pruned and padded as a target imageon the adjacent screen.

Optionally, as another embodiment, when the first width is greater thanthe second width and the source image corresponding to the adjacentscreen of the first screen is pruned as the target image on the adjacentscreen, a part that has a width of c may be pruned from a side that isin the source image corresponding to the adjacent screen and is awayfrom the first screen, and an edge that is in the preset color and has awidth of c may be padded into a side that is in the source imagecorresponding to the adjacent screen and is near the first screen, so asto obtain the target image on the adjacent screen, where c=a−b, a is thefirst width, and b is the second width.

The pruning a part that has a width of c from a side that is in thesource image corresponding to the adjacent screen and is away from thefirst screen may specifically be: determining, in the color array of thesource image, a pixel area corresponding to the part that has a width ofc and is on the side that is in the source image corresponding to theadjacent screen and is away from the first screen; and discarding pixeldata in the pixel area. The padding an edge that is in the preset colorand has a width of c into a side that is in the source imagecorresponding to the adjacent screen and is near the first screen mayspecifically be: determining, in the color array of the source image, aposition of a pixel area corresponding to the part that has a width of cand is on the side that is in the source image corresponding to theadjacent screen and is near the first screen; and adding pixel data ofthe preset color into the pixel area.

Optionally, as another embodiment, when the first width is less than thesecond width and the source image corresponding to the adjacent screenof the first screen is pruned as the target image on the adjacentscreen, a part that has a width of c may be pruned from a side that isin the source image corresponding to the adjacent screen and is near thefirst screen, and an edge that is in the preset color and has a width ofc may be padded into a side that is in the source image corresponding tothe adjacent screen and is away from the first screen, so as to obtainthe target image on the adjacent screen, where c=b−a, a is the firstwidth, and b is the second width.

The pruning a part that has a width of c from a side that is in thesource image corresponding to the adjacent screen and is near the firstscreen may specifically be: determining, in the color array of thesource image, a pixel area corresponding to the part that has a width ofc and is on the side that is in the source image corresponding to theadjacent screen and is near the first screen; and discarding pixel datain the pixel area. The padding an edge that is in the preset color andhas a width of c into a side that is in the source image correspondingto the adjacent screen and is away from the first screen mayspecifically be: determining, in the color array of the source image, aposition of a pixel area corresponding to the part that has a width of cand is on the side that is in the source image corresponding to theadjacent screen and is away from the first screen; and adding pixel dataof the preset color into the pixel area.

Optionally, as another embodiment, when the source image correspondingto the first screen is pruned as the target image on the first screen,an edge that has a width off may be pruned from two sides of the sourceimage corresponding to the first screen, and an edge that is in thepreset color and has a width off may be separately padded into the twosides of the source image corresponding to the first screen, wheref<=|a−b|. and preferably, f=|a−b|/2 or f=|a−b|.

The pruning an edge that has a width of f from two sides of the sourceimage corresponding to the first screen may specifically be:determining, in the color array of the source image, a pixel areacorresponding to the part that has a width of f and is on the two sidesof the source image corresponding to the first screen; and discardingpixel data in the pixel area. The padding an edge that is in the presetcolor and has a width off separately into the two sides of the sourceimage corresponding to the first screen may specifically be:determining, in the color array of the source image, a position of apixel area corresponding to the part that has a width of f and is on thetwo sides of the source image corresponding to the first screen; andadding pixel data of the preset color into the pixel area.

Optionally, as another embodiment, when the first width is greater thanthe second width and the source image corresponding to one or morescreens among the N screens is zoomed out and padded to obtain thetarget image, the source image is zoomed out according to a zoom factork, where k=(b+d)/(a+d), a is the first width, b is the second width, andd is the width of the source image; and an edge in the preset color ispadded around the source image after zooming out, so as to obtain thetarget image. A position of a central point of an image may remainunchanged during the zooming.

The zooming out the source image according to a zoom factor k mayspecifically be: compressing all pixels in the color array of the sourceimage according to the zoom factor k, where the compression mayspecifically be deleting pixel data of a part of pixel points accordingto the zoom factor. The padding an edge in a preset color around thesource image after zooming out specifically is: calculating a part thatneeds to be padded around the source image after zooming out;determining, in the color array of the source image, a position of apixel area corresponding to the part that needs to be padded; and addingpixel data of the preset color into the pixel area.

Optionally, as another embodiment, when the first width is less than thesecond width and the source image corresponding to one or more screensamong the N screens is zoomed in and padded to obtain the target image,the source image is zoomed in according to a zoom factor k, wherek=(b+d)/(a+d), a is the first width, b is the second width, and d is thewidth of the source image; and an edge part is pruned from the sourceimage after zooming in, so as to obtain the target image. The edge partis specifically a part that is in the source image after zooming in andis beyond the scope of the target image. A position of a central pointof an image may remain unchanged during the zooming.

The zooming in the source image according to a zoom factor k mayspecifically be: stretching all pixels in the color array of the sourceimage according to the zoom factor k, where the stretching mayspecifically be adding, between adjacent pixel data, pixel data of atransitive color according to the zoom factor. The pruning the edge partfrom the source image after zooming in may specifically be: calculatingthe edge part of the source image after zooming in; determining, in thecolor array of the source image, a pixel area corresponding to the edgepart; and discarding pixel data in the pixel area.

Optionally, as another embodiment, the preset color may be a frame colorof a display device of the target image or be close to a frame color ofa display device of the target image, for example, if the frame color ofthe display device is RGB (125, 125, 125), the preset color may be setto (125, 125, 125), (120, 120, 120), or the like.

Optionally, as another embodiment, before step 202, a manner foradjusting the source image may be determined according to a presettingor through a user instruction. For example, multiple adjusting mannersmay be provided for the user to select, and one of the adjusting mannersis selected according to the preference of the user. For example, theadjusting manners may include but are not limited to “pruning andpadding an image”, “zooming in or out an image”, and the like.

In addition, preferably, in the embodiment of the present invention, asource image and a target image have same dimensions, that is, it isrequired that both the width and the height of a source image are thesame as those of a target image. A source image may be an original imagethat is actually transmitted during a video conference and may also bean image obtained after an original image is preprocessed. When both thewidth and the height of an original image are the same as those of atarget image, the original image may be directly taken as the sourceimage; otherwise, the original image needs to be preprocessed to obtainthe source image that meets the preceding requirement.

When an original image is preprocessed, the original image may be zoomedin or out, pruned, and/or padded so as to obtain a source image havingthe same dimensions as the target image. The following describes amanner for preprocessing an original image according to the embodimentof the present invention with reference to specific examples. However,these examples are only used to help a person skilled in the art tobetter understand the embodiment of the present invention rather thanlimiting the scope of the embodiment of the present invention.

Optionally, as another embodiment, original images corresponding to theN screens may further be preprocessed to obtain source images thatcorrespond to the N screens and have the same dimensions as the targetimages, where dimensions of the original images may be the same as ordifferent from those of the target images.

Optionally, as another embodiment, if the dimensions of the originalimages are the same as those of the target images, the original imagesare taken as the source images. In this case, it may be determined instep 201 that the first width a is the width A (a=A) of a blind spotbetween the original images.

In another aspect, if the dimensions of the original images aredifferent from those of the target images, the original imagescorresponding to the N screens are separately adjusted to obtaincorresponding source images A manner for adjusting an original image toobtain a source image includes but is not limited to pruning, padding,or zooming in or out the original image.

The pruning the original image may specifically be: calculating a partthat needs to be pruned from the original image; determining, in a colorarray of the original image, a pixel area corresponding to the part thatneeds to be pruned; and discarding pixel data in the pixel area. Thepadding the original image may specifically be: calculating a part thatneeds to be padded; determining, in the color array of the originalimage, a position of a pixel area corresponding to the part that needsto be padded; and adding pixel data of a preset color into the pixelarea corresponding to the position of the pixel area, where the pixeldata of the preset color may specifically be RGB (125, 125, 125). Thezooming out the original image may specifically be: calculating a zoomfactor according to the width of the original image and the width of atarget image; and when the width of the original image is greater thanthe width of the target image, compressing all pixels in a color arrayof the original image according to the zoom factor, where thecompression may specifically be deleting pixel data of a part of pixelpoints according to the zoom factor. The zooming in the original imagemay specifically be: calculating a zoom factor according to the width ofthe original image and the width of a target image; and when the widthof the original image is less than the width of the target image,stretching all pixels in the color array of the original image accordingto the zoom factor, where the stretching may specifically be addingpixel data of a transitive color between adjacent pixel data accordingto the zoom factor, for example, adding, between two adjacent pixelpoints with pixel values of RGB (11, 11, 11) and RGB(33, 33, 33), a newpixel point with pixel data of RGB (22, 22, 22).

For example, if aspect ratios of the original images are the same asthose of the target images, but the width w of the original images isdifferent from the width d of the target images, the original imagescorresponding to the N screens are proportionally zoomed in or outaccording to a zoom factor m so as to obtain corresponding sourceimages, where m=d/w. In this case, it may be determined in step 201 thatthe first width a=n×A, where n=m, and A is the width of a blind spotbetween the original images corresponding to the N screens.

Specifically, FIG. 3A is a schematic diagram of an example of obtainingsource images from original images. In the example of FIG. 3A, theoriginal images have a larger size than target images, but they have asame aspect ratio. For brevity, display devices are omitted in theexample of FIG. 3A, and only outlines of stitched images areillustrated.

As shown in FIG. 3A, original images 105 a, 105 b, and 105 c are at aheight of h and at a width of w. If it is unnecessary to distinguishthem from each other, the original images 105 a, 105 b, and 105 c may becollectively called original images 105 below. The width of a blind spotbetween the original images 105 is A. The width A of a blind spot equalsthe width of a gap between virtual display devices of the originalimages 105, and therefore the original images 105 may be stitchedwithout a mismatch.

Target images 107 a-c may be the same as the target images in FIG.1A/1B. The target images 107 are at a height of e and at a width of d,and the width of a gap between display devices is b.

In the embodiment of FIG. 3A, aspect ratios of the original images 105equal aspect ratios of the target images 107, that is, w/h=d/e. In thiscase, the original images 105 need to be proportionally zoomed in or outso as to obtain corresponding source images 106 a-c. The proportionalzooming refers to zooming according to a same ratio of the height of animage to the width of the image. Specifically, a zoom factor ism=e/h=d/w, and therefore, the width of a blind spot between the sourceimages 106 is a=m×A. In the embodiment of the present invention, thesource images 106 are adjusted based on the width a of a blind spotbetween the source images 106.

The embodiment of FIG. 3A gives an example where the original images 105are larger than the target images 107 and the width a is greater thanthe width b. However, the embodiment of the present invention is notlimited to this, a case where the original images 105 are smaller thanthe target images or the width a is less than the width b is alsoapplicable to the embodiment of the present invention, that is, it isobtained that the width of a blind spot between the source images 106 isa=m×A, where a zoom factor of the width of a blind spot is n=m=e/h=d/w.

For another example, if aspect ratios of the original images aredifferent from those of the target images, the original imagescorresponding to the N screens are proportionally zoomed in or outaccording to a zoom factor m1, so as to obtain correspondingintermediate images, where m1=e/h, e is the height of the target images,and h is the height of the original images. Then, an edge that has atotal width of |d−(ew/h)| is pruned from or padded into one or two sidesof the intermediate images corresponding to the N screens so as toobtain corresponding source images, where d is the width of the targetimages, w is the width of the original images, and ∥ is a calculation oftaking an absolute value.

In this case, it may be determined in step 201 that the first widtha=n×A, where n=(2Ae+ew−dh)/2Ah, and A is the width of a blind spotbetween the original images corresponding to the N screens.

Specifically, FIG. 3B is a schematic diagram of another example ofobtaining source images from original images. In the example of FIG. 3B,the original images have a larger size and a greater aspect ratio(w/h>d/e) than target images. For brevity, display devices are omittedin the example of FIG. 3B, and only outlines of stitched images areillustrated.

As shown in FIG. 3B, firstly, original images 105 are proportionallyzoomed in or out to obtain intermediate images 105 a′-c′ (referred to asintermediate images 105′ below). A zoom factor m1=e/h, where e is theheight of the target images, and h is the height of the original images.Therefore, the width of the intermediate images 105′ is w1=m1×w=ew/h,and the width of a blind spot between the intermediate images 105′ isA1=m1×A=Ae/h. Here, if the height h of the original images 105 is h=e,the original images 105 may also be directly taken as the intermediateimages 105′ (that is, equivalent to m1=1).

As the width w1 of the intermediate images 105′ is still greater thanthe width d of the target images, the intermediate images 105′ may bepruned to obtain source images 106. Specifically, as shown by thedot-shaded part in FIG. 3B, an edge that has a width of x is pruned fromthe left side of an intermediate image 105 a′ to obtain a source image106 a, an edge that has a width of x is pruned from the right side of anintermediate image 105 c′ to obtain a source image 106 c, and an edgethat has a width of x/2 is separately pruned from two sides of anintermediate image 105 b′ to obtain a source image 106 b, wherex=w1−d=(ew/h)−d. In this way, dimensions of the source images 106 arethe same as those of target images 107, and this can simultaneously meeta requirement for stitching without a mismatch.

Therefore, the width of a blind spot between the source images 106 isa=A1+x/2=(2Ae+ew−dh)/2h, that is, a zoom factor of the width of a blindspot is n=a/A=(2Ae+ew−dh)/2Ah. In the embodiment of the presentinvention, the source images 106 are adjusted based on the width a of ablind spot between the source images 106.

FIG. 3C is a schematic diagram of another example of obtaining sourceimages from original images. In the example of FIG. 3C, the originalimages have a larger size and a smaller aspect ratio (w/h<d/e) thantarget images. For brevity, display devices are omitted in the exampleof FIG. 3C, and only outlines of stitched images are illustrated.

A manner for obtaining intermediate images 105′ from original images 105in the embodiment of FIG. 3C is the same as that in the embodiment ofFIG. 3B, and therefore no further details are provided herein. The widthof the intermediate images 105′ is w1=m1×w=ew/h, and the width of ablind spot between the intermediate images 105′ is A1=m1×A=Ae/h.

As the width w1 of the intermediate images 105′ is less than the width dof the target images, the intermediate images 105′ may be padded toobtain source images 106. Specifically, as shown by the grid-shaded partin FIG. 3C, an edge that has a width of y is padded into a left side ofan intermediate image 105 a′ to obtain a source image 106 a, an edgethat has a width of y is padded into a right side of an intermediateimage 105 c′ to obtain a source image 106 c, and an edge that has awidth of y/2 is separately padded into two sides of an intermediateimage 105 b′ to obtain a source image 106 b, where y=d−w1=d−(ew/h). Inthis way, dimensions of the source images 106 are the same as those oftarget images 107, and this can simultaneously meet a requirement forstitching without a mismatch.

Therefore, the width of a blind spot between the source images 106 isa=A1−y/2=(2Ae+ew−dh)/2h, that is, a zoom factor of the width of a blindspot is n=a/A=(2Ae+ew−dh)/2Ah. In the embodiment of the presentinvention, the source images 106 are adjusted based on the width a of ablind spot between the source images 106.

According to the methods in FIG. 3A-FIG. 3C, source images having thesame dimensions as target images can be obtained, and it is ensured thatthe source images are stitched without a mismatch as the originalimages. That is, a gradient of a straight line determined by two actualscenario points that are separately displayed on two original imagescorresponding to adjacent screens is identical to a gradient of astraight line determined by the two actual scenario points that areseparately displayed on two source images corresponding to adjacentscreens.

The following describes in further detail the embodiment of the presentinvention with reference to specific examples. It should be noted thatthese examples are only used to help a person skilled in the art tobetter understand the embodiment of the present invention rather thanlimiting the scope of the embodiment of the present invention. Forexample, in FIG. 4-FIG. 10, examples where source images and targetimages have the same dimensions and N=3 are illustrated. However, theembodiment of the present invention is not limited to this and may besimilarly applied to a case where N=2 or N>3. This application alsofalls within the scope of the embodiment of the present invention. Inaddition, for brevity, display devices are not illustrated in thefollowing embodiments.

FIG. 4A is a schematic diagram of an image processing manner accordingto an embodiment of the present invention. The embodiment of FIG. 4A isapplied to the scenario shown in FIG. 1A, where no processing isperformed on an intermediate screen, that is, a source image 301 bcorresponding to the intermediate screen is directly taken as a targetimage 302 b of the intermediate screen. Only source images 301 a and 301c that correspond respectively to left and right screens (that is,screens adjacent to the intermediate screen) are adjusted to obtaincorresponding target images 302 a and 302 c. It is assumed that thewidth of both the source images and the target images is d.

The upper part of FIG. 4A is a schematic diagram of source images 301.The width of a blind spot between the source images 301 is a first widtha. The lower part of FIG. 4A is a schematic diagram of target images302. The width of a gap between display devices of the target images 302is a second width b, and a>b.

The source images 301 a and 301 c corresponding to the adjacent screensmay be moved by a width of c toward sides that are near the intermediatescreen, that is, an edge that has a width of c is pruned from sides thatare in the source images 301 a and 301 c and are away from theintermediate screen, and an edge that is in a preset color and has awidth of c is padded into sides that are in the source images 301 a and301 c corresponding to the adjacent screens and are near theintermediate screen, so as to obtain the target images 302 a and 302 cof the adjacent screens.

Specifically, firstly, as shown by the dot-shaded areas in FIG. 4A, thesource image 301 a is moved by a width of c rightward, that is, pruningan edge that has a width of c from a left side of the source image 301a; and the source image 301 c is moved by a width of c leftward, thatis, pruning an edge that has a width of c from a right side of thesource image 301 c.

Then, as shown by the grid-shaded areas in FIG. 4A, an edge that has awidth of c is padded into a right side of the pruned source image 301 a,and an edge that has a width of c is padded into a left side of thepruned source image 301 c. In this way, the target images 302 a and 302c are separately obtained.

The target images 302 obtained according to FIG. 4A have the samedimensions as the source images 301. Although effective display areas(non-shaded parts in FIG. 4A) of the target images 302 are smaller thanthose of the source images 301, it is ensured that no mismatch exists institched images 302, and the three slanting lines in FIG. 4A are stillkept in a same straight line. An effective display area is an area thatis in an image and contains conference scene content, for example, anarea that contains personnel who participates in a conference, thebackground, and the like.

Optionally, as an embodiment, a color of a padded edge (a grid-shadedpart in FIG. 4A) may be the same as or close to a frame color of adisplay device, so that a user may visually consider the padded part asa part of the frame of the display device when viewing a target image,and this can enhance visual perception of the user.

In addition, according to the embodiment of FIG. 4A, an image intrue-to-life dimensions can be maintained, thereby enhancing userexperience.

FIG. 4B is a schematic diagram of an image processing manner accordingto another embodiment of the present invention. The embodiment of FIG.4B is applied to the scenario shown in FIG. 1B, where no processing isperformed on an intermediate screen, that is, a source image 301 bcorresponding to the intermediate screen is directly taken as a targetimage 303 b of the intermediate screen. Only source images 301 a and 301c that correspond respectively to left and right screens (that is,screens adjacent to the intermediate screen) are adjusted to obtaincorresponding target images 303 a and 303 c. It is assumed that thewidth of both the source images and the target images is d.

The upper part of FIG. 4B is a schematic diagram of source images 301.The width of a blind spot between the source images 301 is a first widtha. The lower part of FIG. 4B is a schematic diagram of target images303. The width of a gap between display devices of the target images 303is a second width b, and a<b.

The source images 301 a and 301 c corresponding to the adjacent screensmay be moved by a width of c (c=b−a) toward sides that are away from theintermediate screen, that is, an edge that has a width of c is prunedfrom sides that are in the source images 301 a and 301 c and are nearthe intermediate screen, and an edge that is in a preset color and has awidth of c is padded into sides that are in the source images 301 a and301 c corresponding to the adjacent screens and are away from theintermediate screen, so as to obtain the target images 303 a and 303 cof the adjacent screens.

Specifically, firstly, as shown by the dot-shaded areas in FIG. 4B, thesource image 301 a is moved by a width of c leftward, that is, pruningan edge that has a width of c from a right side of the source image 301a; and the source image 301 c is moved by a width of c rightward, thatis, pruning an edge that has a width of c from a left side of the sourceimage 301 c.

Then, as shown by the grid-shaded areas in FIG. 4B, an edge that has awidth of c is padded into a right side of the pruned source image 301 c,and an edge that has a width of c is padded into a left side of thepruned source image 301 a. In this way, the target images 303 a and 303c are obtained.

The target images 303 obtained according to FIG. 4B have the samedimensions as the source images 301. Although effective display areas(non-shaded parts in FIG. 4B) of the target images 303 are smaller thanthose of the source images 301, it is ensured that no mismatch exists institched images 303, and the three slanting lines in FIG. 4B are stillkept in a same straight line.

Optionally, as an embodiment, a color of a padded edge (a grid-shadedpart in FIG. 4B) may be the same as or close to a frame color of adisplay device, so that a user may visually consider the padded part asa part of the frame of the display device when viewing a target image,and this can enhance visual perception of the user.

In addition, according to the embodiment of FIG. 4B, an image intrue-to-life dimensions can be maintained, thereby enhancing userexperience.

FIG. 5 is a schematic diagram of an image processing manner according toanother embodiment of the present invention. The embodiment of FIG. 5 isa variation on the embodiment of FIG. 4A and is applied to the scenarioshown in FIG. 1A, where a source image 301 b corresponding to anintermediate screen is also pruned.

Specifically, a method for obtaining a target image 304 a in FIG. 5 isthe same as the method for obtaining the target image 302 a in FIG. 4A,and a method for obtaining a target image 304 c in FIG. 5 is the same asthe method for obtaining the target image 302 c in FIG. 4A. Therefore,no further details are provided herein.

As shown by the slant-shaded parts in FIG. 5, an edge that has a widthof f is separately pruned from two sides of the source image 301 bcorresponding to the intermediate screen, and an edge that is in apreset color and has a width of f is then padded into the pruned parts,so as to obtain a target image 304 b of the intermediate screen.Optionally, the preset color of the padded edge may be the same as orclose to a frame color of a display device, and this can enhance visualperception of a user.

In addition, in the example of FIG. 5, the width f=|a−b|/2. As a firstwidth a is greater than a second width b, f=(a−b)/2.

According to the embodiment of FIG. 5, effective display areas(non-shaded parts in FIG. 5) of the three target images 304 a, 304 b,and 304 c have the same dimensions, and an edge that has a total widthof c is separately pruned on the basis of corresponding source images.This can enhance user experience.

FIG. 6 is a schematic diagram of an image processing manner according toanother embodiment of the present invention. The embodiment of FIG. 6 isa variation on the embodiment of FIG. 4B and is applied to the scenarioshown in FIG. 1B, where a source image 301 b corresponding to anintermediate screen is also pruned.

Specifically, a method for obtaining a target image 305 a in FIG. 6 isthe same as the method for obtaining the target image 303 a in FIG. 4B,and a method for obtaining a target image 305 c in FIG. 6 is the same asthe method for obtaining the target image 303 c in FIG. 4B. Therefore,no further details are provided herein.

As shown by the slant-shaded parts in FIG. 6, an edge that has a widthof f is separately pruned from two sides of the source image 301 bcorresponding to the intermediate screen, and an edge that is in apreset color and has a width of f is then padded into the pruned parts,so as to obtain a target image 305 b of the intermediate screen.Optionally, the preset color of the padded edge may be the same as orclose to a frame color of a display device, and this can enhance visualperception of a user.

In addition, in the example of FIG. 6, the width f=|a−b|/2. As a firstwidth a is less than a second width b, f=(b−a)/2.

According to the embodiment of FIG. 6, effective display areas(non-shaded parts in FIG. 6) of the three target images 305 a, 305 b,and 305 c have the same dimensions, and an edge that has a total widthof c is separately pruned on the basis of corresponding source images.This can enhance user experience.

FIG. 7 is a schematic diagram of an image processing manner according toanother embodiment of the present invention. The embodiment of FIG. 7 isa variation on the embodiment of FIG. 4A and is applied to the scenarioshown in FIG. 1A, where a source image 301 b corresponding to anintermediate screen is also pruned.

Specifically, a method for obtaining a target image 306 a in FIG. 7 isthe same as the method for obtaining the target image 302 a in FIG. 4A,and a method for obtaining a target image 306 c in FIG. 7 is the same asthe method for obtaining the target image 302 c in FIG. 4A. Therefore,no further details are provided herein.

As shown by the slant-shaded parts in FIG. 7, an edge that has a widthof f is separately pruned from two sides of the source image 301 bcorresponding to the intermediate screen, and an edge that is in apreset color and has a width of f is then padded into the pruned parts,so as to obtain a target image 306 b of the intermediate screen.Optionally, the preset color of the padded edge may be the same as orclose to a frame color of a display device, and this can enhance visualperception of a user.

In addition, in the example of FIG. 7, the width f=|a−b|. As a firstwidth a is greater than a second width b, f=a−b=c.

According to the embodiment of FIG. 7, distances between edges ofeffective display areas (non-shaded parts in FIG. 7) of the three targetimages 306 a, 306 b, and 306 c and frames of display devices are thesame, that is, a−b. This can enhance user experience.

FIG. 8 is a schematic diagram of an image processing manner according toanother embodiment of the present invention. The embodiment of FIG. 8 isa variation on the embodiment of FIG. 4B and is applied to the scenarioshown in FIG. 1B, where a source image 301 b corresponding to anintermediate screen is also pruned.

Specifically, a target image 307 a in FIG. 8 is the same as the targetimage 303 a in FIG. 4B, and a target image 307 c in FIG. 8 is the sameas the target image 303 c in FIG. 4B. Therefore, no further details areprovided herein.

As shown by the slant-shaded parts in FIG. 8, an edge that has a widthof f is separately pruned from two sides of the source image 301 bcorresponding to the intermediate screen, and an edge that is in apreset color and has a width of f is then padded into the pruned parts,so as to obtain a target image 307 b of the intermediate screen.Optionally, the preset color of the padded edge may be the same as orclose to a frame color of a display device, and this can enhance visualperception of a user.

In addition, in the example of FIG. 8, the width f=|a−b|. As a firstwidth a is less than a second width b, f=b−a=c.

According to the embodiment of FIG. 8, distances between edges ofeffective display areas (non-shaded parts in FIG. 7) of the targetimages 307 a, 307 b, and 307 c and frames of display devices are thesame, that is, b−a. This can enhance user experience.

FIG. 9 is a schematic diagram of an image processing manner according toanother embodiment of the present invention. The embodiment of FIG. 9 isapplied to the scenario shown in FIG. 1A, where, when a source imagecorresponding to one or more screens among N screens is zoomed in or outto obtain a target image, the source image may be zoomed out accordingto a zoom factor k, k=(b+d)/(a+d), a is a first width, b is a secondwidth, d is the width of the source image, and a>b; and an edge in apreset color is padded around the source image after zooming out, so asto obtain the target image. A position of a central point of an imagemay remain unchanged during the zooming.

Specifically, as shown in FIG. 9, it is assumed that source images 301a-301 c are at a width of d and at a height of e, and a zoom ratio is k.Therefore, the part after zooming out is at a width of kd and at aheight of ke. If it is still required that no mismatch exists in imagesafter zooming out, that is, as shown in FIG. 9, gradients of slantinglines in source images and target images remain unchanged, equation (1)may be obtained:[3kd+2b+4(1−k)d/2]/ke=(3d+2a)/e  (1)

It may be obtained according to equation (1) that:k=(b+d)/(a+d)  (2)

Therefore, the source images 301 a-301 c need to be zoomed in or out to(b+d)/(a+d) of the original sizes. In this embodiment, the zoom factork<1, and therefore one frame (a slant-shaded part shown in FIG. 9) needsto be padded around each image after zooming out, so as to obtain targetimages 308 a-308 c. If parts (non-shaded parts shown in the lower partof FIG. 9) of source images after zooming out are still in the middle,it can be obtained that the sum of the height of upper and lower framesis e(a−b)/(a+d), and the sum of the width of left and right frames isd(a−b)/(a+d). Generally the height of the upper and lower frames may bethe same, that is, e(a−b)/2(a+d); and the width of the left and rightframes is also the same, that is, d(a−b)/2(a+d).

However, the embodiment of the present invention is not limited to this.The source images after zooming out may also not in the middle of thetarget images. For example, parts of the source images after zooming outmay be simultaneously moved by a same distance downward on the basis ofthe target images in FIG. 9, including being moved to lower edges ofdisplay devices; or parts of the source images after zooming out may besimultaneously moved by a same distance upward on the basis of thetarget images in FIG. 9, including being moved to upper edges of displaydevices; or parts of the source images after zooming out may besimultaneously moved by a same distance leftward on the basis of thetarget images in FIG. 9, including being moved to left edges of displaydevices; or parts of the source images after zooming out may besimultaneously moved by a same distance rightward on the basis of thetarget images in FIG. 9, including being moved to right edges of displaydevices; or the like. All these manners can still ensure that nomismatch exists in the stitched target images 308 a-380 c.

Optionally, the preset color of the padded edge may be the same as orclose to a frame color of a display device, and this can enhance visualperception of a user.

According to the adjusting manner in FIG. 9, no information about sourceimages is lost, and stitching of target images without a mismatch isensured. In addition, effective display areas (non-shaded parts in FIG.9) of the three target images are of a same size, and the userexperience is identical.

FIG. 10 is a schematic diagram of an image processing manner accordingto another embodiment of the present invention. The embodiment of FIG.10 is applied to the scenario shown in FIG. 1B, where, when a sourceimage corresponding to one or more screens among N screens is zoomed inor out and pruned to obtain a target image, the source image may bezoomed in according to a zoom factor k, k=(b+d)/(a+d), a is a firstwidth, b is a second width, d is the width of the source image, and a<b;and an edge part is pruned from the source image after zooming in, so asto obtain the target image. A position of a central point of an imagemay remain unchanged during the zooming.

As the second width b is greater than the first width a, source images301 a-301 c need to be zoomed in so as to ensure that no mismatch existsin stitched target images.

Specifically, as shown by the dashed boxes at the lower part of FIG. 10,the width and the height of the source images after zooming in are kdand ke respectively. In this case, edge parts, that is, dot-shaded partsin FIG. 10, need to be pruned to obtain target images 309 a-309 c.

According to the manner in FIG. 10, stitching of the target images 309a-309 c without a mismatch can also be ensured.

According to the preceding embodiment, the width of a blind spot betweensource images and the width of a gap between display devices of targetimages may be acquired, and the source images may be adjusted accordingto the acquired width values so as to obtain stitched target imageswithout a mismatch, thereby improving a sense of reality oftelepresence.

FIG. 11 is a block diagram of an image processing device according to anembodiment of the present invention. The image processing device 110 inFIG. 11 includes a determining unit 111 and an adjusting unit 112.

The determining unit 111 determines a first width and a second width,where the second width is the width of a gap between display devices oftarget images on N screens, the first width is the width of a blind spotbetween source images corresponding to the N screens, N is an integergreater than 1, and the N screens are of a same size and are arrangedside by side at a same height.

The adjusting unit 112 adjusts, when the first width is different fromthe second width, the source images according to the first width and thesecond width that are determined by the determining unit 111, so as toobtain the target images, so that no mismatch exists in the stitchedtarget images on the N screens.

Stitching target images without a mismatch ensures that a gradient (forexample, a gradient relative to a horizontal line) of a straight linedetermined by two actual scenario points that are separately displayedon two source images on adjacent screens is identical to a gradient of astraight line determined by the two actual scenario points that areseparately displayed on two target images on adjacent screens.

According to the embodiment of the present invention, the width of ablind spot between source images and the width of a gap between displaydevices of target images may be acquired, and the source images may beadjusted according to the acquired width values so as to obtain stitchedtarget images without a mismatch, thereby improving a sense of realityof telepresence.

The image processing device 110 according to the embodiment of thepresent invention is capable of implementing the method shown in FIG. 2and each embodiment shown in FIG. 3-FIG. 10. To avoid repetition, nofurther details are provided herein.

Optionally, as an embodiment, the adjusting unit 112 may prune a sourceimage corresponding to one or more screens among the N screens so as toobtain a target image; or zoom in or out a source image corresponding toone or more screens among the N screens so as to obtain a target image;or zoom in or out and prune a source image corresponding to one or morescreens among the N screens so as to obtain a target image.

Optionally, as another embodiment, the adjusting unit 112 may take asource image corresponding to a first screen among the N screens as atarget image on the first screen, or prune a source image correspondingto a first screen as a target image on the first screen; and prune asource image corresponding to an adjacent screen of the first screen asa target image on the adjacent screen.

Optionally, as another embodiment, the adjusting unit 112 may calculatea part that needs to be pruned from the source image, determine, in acolor array of the source image, a pixel area corresponding to the partthat needs to be pruned, and discard pixel data in the pixel area; andcalculate a part that needs to be padded, determine, in the color arrayof the source image, a position of a pixel area corresponding to thepart that needs to be padded, and add pixel data of a preset color intothe pixel area corresponding to the position of the pixel area, therebyobtaining the target image corresponding to the source image.

Optionally, as another embodiment, the adjusting unit 112 may calculatea zoom factor according to the first width and the second width; andwhen the first width is greater than the second width, compress allpixels in a color array of the source image according to the zoomfactor, calculate a part that needs to be padded, determine, in thecolor array of the source image, a position of a pixel areacorresponding to the part that needs to be padded, and add pixel data ofa preset color into the pixel area corresponding to the position of thepixel area, thereby obtaining the target image corresponding to thesource image.

Optionally, as another embodiment, the adjusting unit 112 may calculatea zoom factor according to the first width and the second width; andwhen the first width is less than the second width, stretch all pixelsin a color array of the source image according to the zoom factor,calculate a part that needs to be pruned from the source image,determine, in the color array of the source image, a pixel areacorresponding to the part that needs to be pruned, and discard pixeldata in the pixel area, thereby obtaining the target image correspondingto the source image.

Optionally, as another embodiment, the adjusting unit 112 may prune apart that has a width of c from a side that is in the source imagecorresponding to the adjacent screen and is away from the first screen,and pad an edge that is in a preset color and has a width of c into aside that is in the source image corresponding to the adjacent screenand is near the first screen, so as to obtain the target image on theadjacent screen, where a is the first width, b is the second width, a>b,and c=a−b. This can enhance user experience, which is exemplarilyillustrated in the embodiment of FIG. 4A.

Optionally, as another embodiment, the adjusting unit 112 may prune apart that has a width of c from a side that is in the source imagecorresponding to the adjacent screen and is near the first screen, andpad an edge that is in a preset color and has a width of c into a sidethat is in the source image corresponding to the adjacent screen and isaway from the first screen, so as to obtain the target image on theadjacent screen, where a is the first width, b is the second width, a<b,and c=b−a. This can enhance user experience, which is exemplarilyillustrated in the embodiment of FIG. 4B.

Optionally, as another embodiment, the adjusting unit 112 may prune anedge that has a width of f from two sides of the source imagecorresponding to the first screen, and separately pad an edge that is ina preset color and has a width of f into the two sides of the sourceimages corresponding to the first screen. This can enhance userexperience, which is exemplarily illustrated in the embodiments of FIG.5-FIG. 8. Optionally, f=|a−b|/2 or f=|a−b|.

Optionally, as another embodiment, the adjusting unit 112 may zoom outthe source image according to a zoom factor k, where k=(b+d)/(a+d), a isthe first width, b is the second width, d is the width of the sourceimage, and a>b, and pad an edge in the preset color around the sourceimage after zooming out, so as to obtain the target image. A position ofa central point of an image may remain unchanged during the zooming.This can enhance user experience, which is exemplarily illustrated inthe embodiment of FIG. 9.

Optionally, as another embodiment, the adjusting unit 112 may zoom inthe source image according to a zoom factor k, where k=(b+d)/(a+d), a isthe first width, b is the second width, d is the width of the sourceimage, and a<b; and prune an edge part from the source image afterzooming in, so as to obtain the target image. A position of a centralpoint of an image may remain unchanged during the zooming. This canenhance user experience, which is exemplarily illustrated in theembodiment of FIG. 10.

Optionally, as another embodiment, the preceding preset color may be aframe color of a display device of a target image or be close to a framecolor of a display device of a target image.

Optionally, as another embodiment, the determining unit 111 may furtherdetermine, according to a presetting or a user instruction, a manner foradjusting a source image, so that the adjusting unit 112 may adjust thesource image according to the determined adjusting manner. For example,multiple adjusting manners may be provided for the user to select, andone of the adjusting manners is selected according to the preference ofthe user.

Optionally, as another embodiment, the adjusting unit 112 may furtherpreprocess original images corresponding to the N screens so as toobtain source images that correspond to the N screens and have the samedimensions as target images, where dimensions of the original images arethe same as or different from those of the target images.

Optionally, as another embodiment, if the dimensions of the originalimages are the same as those of the target images, the adjusting unit112 may take the original images as the source images; and if thedimensions of the original images are different from those of the targetimages, the adjusting unit 112 may separately zoom in or out, prune,and/or pad the original images corresponding to the N screens so as toobtain corresponding source images.

Optionally, as another embodiment, if aspect ratios of the originalimages are the same as those of the target images and the width w of theoriginal images is different from the width d of the target images, theadjusting unit 112 may proportionally zoom in or out, according to azoom factor m, the original images corresponding to the N screens so asto obtain corresponding source images, where m=d/w.

In this case, the determining unit 111 may determine that the firstwidth a=n×A, where n=m, and A is the width of a blind spot between theoriginal images corresponding to the N screens, exemplarily asillustrated in the embodiment of FIG. 3A.

Optionally, as another embodiment, if aspect ratios of the originalimages are different from those of the target images, the adjusting unit112 may proportionally zoom in or out, according to a zoom factor m1,the original images corresponding to the N screens so as to obtaincorresponding intermediate images, where m1=e/h, e is the height of thetarget images, and h is the height of the original images; andseparately zoom in or out, according to a zoom factor m2, the width ofthe intermediate images corresponding to the N screens so as to obtaincorresponding source images, where m2=dh/ew, d is the width of thetarget images, and w is the width of the original images.

In this case, the determining unit 111 may determine that the firstwidth a=n×A, where n=d/w, and A is the width of a blind spot between theoriginal images corresponding to the N screens.

Optionally, as another embodiment, if aspect ratios of the originalimages are different from those of the target images, the adjusting unit112 may proportionally zoom in or out, according to a zoom factor m1,the original images corresponding to the N screens so as to obtaincorresponding intermediate images, where m1=e/h, e is the height of thetarget images, and h is the height of the original images; and prune orpad an edge that has a total width of |d−(ew/h)| from or into one or twosides of the intermediate images corresponding to the N screens so as toobtain corresponding source images, where d is the width of the targetimages, w is the width of the original images, and ∥ is a calculation oftaking an absolute value.

In this case, the determining unit 111 may determine that the firstwidth a=n×A, where n=(2Ae+ew−dh)/2Ah, and A is the width of a blind spotbetween the original images corresponding to the N screens, exemplarilyas illustrated in the embodiments of FIG. 3B and FIG. 3C.

According to the embodiment of the present invention, the imageprocessing device may acquire the width of a blind spot between sourceimages and the width of a gap between display devices of target images,and adjust the source images according to the acquired width values soas to obtain stitched target images without a mismatch, therebyimproving a sense of reality of telepresence. Even if the dimensions oforiginal images are different from those of the target images, theoriginal images may also be preprocessed to obtain the source images,and then the source images are adjusted to obtain the stitched targetimages without a mismatch.

FIG. 12 is a schematic block diagram of an image processing deviceaccording to another embodiment of the present invention. The imageprocessing device 120 in FIG. 12 includes a processor 121 and a memory122.

The memory 122 stores instructions for the processor 121 to execute thefollowing operations: determining a first width and a second width,where the second width is the width of a gap between display devices oftarget images on N screens, the first width is the width of a blind spotbetween source images corresponding to the N screens, N is an integergreater than 1, and the N screens are of a same size and are arrangedside by side at a same height; and when the first width is differentfrom the second width, adjusting the source images according to thedetermined first width and second width so as to obtain the targetimages, so that no mismatch exists in the stitched target images on theN screens.

Stitching target images without a mismatch ensures that a gradient (forexample, a gradient relative to a horizontal line) of a straight linedetermined by two actual scenario points that are separately displayedon two source images on adjacent screens is identical to a gradient of astraight line determined by the two actual scenario points that areseparately displayed on two target images on adjacent screens.

According to the embodiment of the present invention, the width of ablind spot between source images and the width of a gap between displaydevices of target images are acquired, and the source images areadjusted according to the acquired width values so as to obtain stitchedtarget images without a mismatch, thereby improving a sense of realityof telepresence.

The image processing device 120 according to the embodiment of thepresent invention is capable of implementing the method shown in FIG. 2and each embodiment shown in FIG. 3-FIG. 10. To avoid repetition, nofurther details are provided herein.

The processor 121 and the memory 122 are connected through a bus system.Except for a data bus, the bus system may further include a power bus, acontrol bus, a status signal bus, and the like. The embodiment of thepresent invention does not set a limitation to a specific form of thebus.

The memory 122 stores instructions for the processor 121 to execute eachoperation and data required for executing each operation. The processor121 may be a CPU (Central Processing Unit, central processing unit), andmay also be a GPU (Graphic Processing Unit, graphics processing unit),or may be in other forms, such as a general processor, a digital signalprocessor (DSP), an application-specific integrated circuit (ASIC), afield programmable gate array (FPGA) or another programmable logicdevice, a discrete-gate or transistor logic device, and a discretehardware component, and is capable of implementing or executing eachmethod, step, and logical block diagram that are disclosed in theembodiments of the present invention. The general processor may be amicroprocessor, or the processor may also be any common processor or thelike. The memory 122 may include a read-only memory and a random accessmemory, and provide an instruction and data for the processor 121. Apart of the memory 122 may also include a non-volatile random accessmemory (NVRAM).

In addition, in the implementation process, each step in the precedingmethods may be implemented by an integrated logic circuit of hardware orthrough instruction control in a software form, where the hardware andthe software are on the processor 121. The combination of the processor121 and the steps in the methods disclosed in the embodiments of thepresent invention may directly be embodied as being implemented by ahardware decoding processor, or by a combination of hardware andsoftware modules on a decoding processor. A software module may belocated on a mature storage medium in the field, such as a random accessmemory, a flash memory, a read-only memory, a programmable read-onlymemory or an electrically erasable and programmable memory, and aregister. The storage medium is located on the memory 122, and theprocessor 121 reads information on the memory 122 and implements thesteps in the preceding methods by using its hardware.

Optionally, as an embodiment, the first width and the second width maybe extracted by the image processing device 120 according to a systemspecification parameter (for example, a specification parameter of adisplay device) and may also be obtained according to an input of auser. The embodiment of the present invention does not set a limitationto this.

Optionally, as an embodiment, a manner for adjusting the source imagesto obtain the target images includes but is not limited to pruningand/or zooming in or out a source image corresponding to one or morescreens among the N screens.

Optionally, as another embodiment, when pruning a source imagecorresponding to one or more screens among the N screens so as to obtaina target image, the processor 121 may take a source image correspondingto a first screen among the N screens as a target image on the firstscreen, or prune a source image corresponding to a first screen as atarget image on the first screen; and prune a source image correspondingto an adjacent screen of the first screen as a target image on theadjacent screen.

Optionally, as another embodiment, when pruning a source imagecorresponding to an adjacent screen of the first screen as a targetimage on the adjacent screen, the processor 121 may prune a part thathas a width of c from a side that is in the source image correspondingto the adjacent screen and is away from the first screen, and pad anedge that is in a preset color and has a width of c into a side that isin the source image corresponding to the adjacent screen and is near thefirst screen, so as to obtain the target image on the adjacent screen,where a is the first width, b is the second width, a>b, and c=a−b.

Optionally, as another embodiment, when pruning a source imagecorresponding to an adjacent screen of the first screen as a targetimage on the adjacent screen, the processor 121 may prune a part thathas a width of c from a side that is in the source image correspondingto the adjacent screen and is near the first screen, and pad an edgethat is in a preset color and has a width of c into a side that is inthe source image corresponding to the adjacent screen and is away fromthe first screen, so as to obtain the target image on the adjacentscreen, where a is the first width, b is the second width, a<b, andc=b−a.

Optionally, as another embodiment, when pruning a source imagecorresponding to a first screen as a target image on the first screen,the processor 121 may prune an edge that has a width off from two sidesof the source image corresponding to the first screen, and separatelypad an edge that is in a preset color and has a width of f into the twosides of the source image corresponding to the first screen, wheref<=|a−b| and optionally, f=|a−b|/2 or f=|a−b|.

Optionally, as another embodiment, when zooming in or out a source imagecorresponding to one or more screens among the N screens so as to obtaina target image, the processor 121 may zoom out the source imageaccording to a zoom factor k, and k=(b+d)/(a+d), where a is the firstwidth, b is the second width, d is the width of the source image, anda>b; and pad an edge in a preset color around the source image afterzooming out, so as to obtain the target image. A position of a centralpoint of an image may remain unchanged during the zooming.

Optionally, as another embodiment, when zooming in or out and pruning asource image corresponding to one or more screens among the N screens soas to obtain a target image, the processor 121 may zoom in the sourceimage according to a zoom factor k, and k=(b+d)/(a+d), where a is thefirst width, b is the second width, d is the width of the source image,and a<b; and prune an edge part from the source image after zooming in,so as to obtain the target image. A position of a central point of animage may remain unchanged during the zooming.

Optionally, as another embodiment, the preceding preset color may be aframe color of a display device of a target image or be close to a framecolor of a display device of a target image.

Optionally, as another embodiment, the processor 121 may furtherdetermine, according to a presetting or through a user instruction, amanner for adjusting the source image. For example, multiple adjustingmanners may be provided for the user to select, and one of the adjustingmanners is selected according to the preference of the user.

Optionally, as another embodiment, the processor 121 may furtherpreprocess original images corresponding to the N screens so as toobtain source images that correspond to the N screens and have the samedimensions as target images, where dimensions of the original images isthe same as or different from those of the target images.

Optionally, as another embodiment, if the dimensions of the originalimages are the same as those of the target images, the processor 121 maytake the original images as the source images; and if the dimensions ofthe original images are different from those of the target images, theprocessor 121 may separately zoom in or out, prune, and/or pad theoriginal images corresponding to the N screens so as to obtaincorresponding source images.

Optionally, as another embodiment, if aspect ratios of the originalimages are the same as those of the target images and the width w of theoriginal images is different from the width d of the target images, theprocessor 121 may proportionally zoom in or out, according to a zoomfactor m, the original images corresponding to the N screens, so as toobtain corresponding source images, where m=d/w.

In this case, the processor 121 may determine that the first widtha=n×A, where n=m, and A is the width of a blind spot between theoriginal images corresponding to the N screens, exemplarily asillustrated in the embodiment of FIG. 3A.

Optionally, as another embodiment, if aspect ratios of the originalimages are different from those of the target images, the processor 121may: proportionally zoom in or out, according to a zoom factor m1, theoriginal images corresponding to the N screens so as to obtaincorresponding intermediate images, where m1=e/h, e is the height of thetarget images, and h is the height of the original images; andseparately zoom in or out, according to a zoom factor m2, the width ofthe intermediate images corresponding to the N screens so as to obtaincorresponding source images, where m2=dh/ew, d is the width of thetarget images, and w is the width of the original images.

In this case, the processor 121 may determine that the first widtha=n×A, where n=d/w, and A is the width of a blind spot between theoriginal images corresponding to the N screens.

Optionally, as another embodiment, if aspect ratios of the originalimages are different from those of the target images, the processor 121may: proportionally zoom in or out, according to a zoom factor m1, theoriginal images corresponding to the N screens, so as to obtaincorresponding intermediate images, where m1=e/h, e is the height of thetarget images, and h is the height of the original images; and prune orpad an edge that has a total width of |d−(ew/h)| from or into one or twosides of the intermediate images corresponding to the N screens, so asto obtain corresponding source images, where d is the width of thetarget images, w is the width of the original images, and ∥ is acalculation of taking an absolute value.

In this case, the processor 121 may determine that the first widtha=n×A, where n=(2Ae+ew−dh)/2Ah, and A is the width of a blind spotbetween the original images corresponding to the N screens, exemplarilyas illustrated in the embodiments of FIG. 3B and FIG. 3C.

According to the embodiment of the present invention, the width of ablind spot between source images and the width of a gap between displaydevices of target images may be acquired, and the source images may beadjusted according to the acquired width values so as to obtain stitchedtarget images without a mismatch, thereby improving a sense of realityof telepresence. Even if the dimensions of original images are differentfrom those of the target images, the original images may also bepreprocessed to obtain the source images, and then the source images areadjusted to obtain the stitched target images without a mismatch.

In addition, embodiments of the present invention do not set alimitation to positions of the image processing devices 110 and 120. Theimage processing devices 110 and 120 may be located at a shooting end ofa remote conference site, may also be located at a target end fordisplaying a target image, or may be located between a shooting end anda target end. The image processing devices 110 and 120 each may be anindependent device and may also act as a part of a device.

A person of ordinary skill in the art may be aware that, in combinationwith the examples described in the embodiments disclosed in thisspecification, units and algorithm steps may be implemented byelectronic hardware, or a combination of computer software andelectronic hardware. Whether the functions are performed by hardware orsoftware depends on particular applications and design constraintconditions of the technical solutions. A person skilled in the art mayuse different methods to implement the described functions for eachspecific application, but it should not be considered that theimplementation goes beyond the scope of the present invention.

It may be clearly understood by a person skilled in the art that, forthe purpose of convenient and brief description, for a detailed workingprocess of the foregoing system, apparatus, and unit, reference may bemade to a corresponding process in the foregoing method embodiments, anddetails are not described herein again.

In the several embodiments provided in the present application, itshould be understood that the disclosed system, apparatus, and methodmay be implemented in other manners. For example, the describedapparatus embodiment is merely exemplary. For example, the unit divisionis merely logical function division and may be other division in actualimplementation. For example, a plurality of units or components may becombined or integrated into another system, or some features may beignored or not performed. In addition, the displayed or discussed mutualcouplings or direct couplings or communication connections may beimplemented through some interfaces. The indirect couplings orcommunication connections between the apparatuses or units may beimplemented in electronic, mechanical or other forms.

The units described as separate parts may or may not be physicallyseparate, and parts displayed as units may or may not be physical units,may be located in one position, or may be distributed on a plurality ofnetwork units. A part or all of the units may be selected according toactual needs to achieve the objectives of the solutions of theembodiments.

In addition, functional units in the embodiments of the presentinvention may be integrated into one processing unit, or each of theunits may exist alone physically, or two or more units are integratedinto one unit.

When the functions are implemented in a form of a software functionalunit and sold or used as an independent product, the functions may bestored in a computer-readable storage medium. Based on such anunderstanding, the technical solutions of the present inventionessentially, or the part contributing to the prior art, or part of thetechnical solutions may be implemented in a form of a software product.The computer software product is stored in a storage medium, andincludes several instructions for instructing a computer device (whichmay be a personal computer, a server, or a network device or the like)to perform all or a part of the steps of the methods described in theembodiments of the present invention. The foregoing storage mediumincludes: any medium that can store program codes, such as a USB flashdisk, a removable hard disk, a read-only memory (ROM, Read-Only Memory),a random access memory (RAM, Random Access Memory), a magnetic disk, oran optical disk.

The foregoing descriptions are merely specific embodiments of thepresent invention, but are not intended to limit the protection scope ofthe present invention. Any variation or replacement readily figured outby a person skilled in the art within the technical scope disclosed inthe present invention shall fall within the protection scope of thepresent invention. Therefore, the protection scope of the presentinvention shall be subject to the protection scope of the claims.

What is claimed is:
 1. An image processing method for providing videoconferencing, comprising: determining, by a processor, a first widthcorresponding to width of one or more blind spots between N sourceimages and determining a second width corresponding to width of one ormore gaps between N target display devices, wherein N is an integergreater than 1 and the N target display devices have screens that are ofa same size and are arranged side by side at a same height, and whereinthe first width is different from the second width; and adjusting, bythe processor, the source images according based on the determined firstwidth and second width so as to obtain target images to be displayed onthe N target display devices such that no mismatch exists in the targetimages displayed on the N target display devices; wherein the adjustingcomprises pruning and padding a source image so as to obtain acorresponding target image; wherein the pruning and padding comprises:calculating a part that needs to be pruned from the source image;determining in a color array of the source image a pixel areacorresponding to the part that needs to be pruned; discarding pixel datain the pixel area; calculating a part that needs to be padded;determining in the color array of the source image a position of a pixelarea corresponding to the part that needs to be padded; and adding pixeldata of a preset color into the pixel area corresponding to the positionof the pixel area thereby obtaining the target image corresponding tothe source image.
 2. The method according to claim 1, wherein the presetcolor is a frame color of a display device of the target image.
 3. Themethod according to claim 1, wherein the pruning and padding comprises:taking a first source image as a target image on a first target displaydevice or pruning and padding the first source image as a target imageon the first target display device; and pruning and padding a secondsource image adjacent to the first source image to obtain a target imageon a second target display device adjacent to the first target displaydevice.
 4. The method according to claim 3, wherein pruning and paddingthe second source image comprises: when the first width is greater thanthe second width, pruning a part that has a width of c from a side thatis in the second source image away from the first source image, andpadding an edge that is in a preset color and has a width of c into aside that is in the second source image near the first source image, soas to obtain the target image for the second target display device,wherein c=a−b, a is the first width, and b is the second width.
 5. Themethod according to claim 3, wherein the pruning and padding the secondsource image comprises: when the first width is less than the secondwidth, pruning a part that has a width of c from a side that is in thesecond source image near the first source image, and padding an edgethat is in a preset color and has a width of c into a side that is inthe second source image away from the first source image, so as toobtain the target image for the second target display device, whereinc=b−a, a is the first width, and b is the second width.
 6. The methodaccording to claim 3, wherein the pruning and padding the first sourceimage comprises: pruning an edge that has a width of f from two sides ofthe first source image, and separately padding an edge that is in apreset color and has a width off into the two sides of the first sourceimage, wherein f<=|a−b|, a is the first width, and b is the secondwidth.
 7. The method according to claim 6, wherein f=|a−b|/2 or f=|a−b|.8. An image processing method for providing video conferencing,comprising: determining a first width corresponding to width of one ormore blind spots between N source images and determining a second widthcorresponding to width of one or more gaps between N target displaydevices wherein N is an integer greater than 1 and the N target displaydevices have screens that are of a same size and are arranged side byside at a same height, and wherein the first width is different from thesecond width; and adjusting the source images according based on thedetermined first width and second width so as to obtain target images tobe displayed on the N target display devices such that no mismatchexists in the tar et images displayed on the N tar et display devices;wherein the adjusting comprises zooming out and padding a source imageso as to obtain a corresponding target image; wherein the zooming outand padding comprises: calculating a zoom factor according to the firstwidth and the second width; and when the first width is greater than thesecond width compressing all pixels in a color array of the source imageaccording to the zoom factor, calculating a part that needs to be paddeddetermining in the color array of the source image a of a pixel areacorresponding to the part that needs to be padded and adding pixel dataof a preset color into the pixel area corresponding to the position ofthe pixel area, thereby obtaining the target image corresponding to thesource image.
 9. The method according to claim 8, wherein the zoomingout and padding comprises: when the first width is greater than thesecond width, zooming out the source image according to a zoom factor k,wherein k=(b+d)/(a+d), a is the first width, b is the second width, andd is the width of the source image; and padding an edge in the presetcolor around the source image after zooming out, so as to obtain thetarget image corresponding to the source image.
 10. An image processingmethod for providing video conferencing, comprising: determining a firstwidth corresponding to width of one or more blind spots between N sourceimages and determining a second width corresponding to width of one ormore gaps between N target display devices wherein N is an integergreater than 1 and the N target display devices have screens that are ofa same size and are arranged side by side at a same height, and whereinthe first width is different from the second width; and adjusting thesource images according based on the determined first width and secondwidth so as to obtain target images to be displayed on the N targetdisplay devices such that no mismatch exists in the tar et imagesdisplayed on the N target display devices; wherein the adjustingcomprises zooming in and pruning a source image so as to obtain acorresponding target image; wherein the zooming and pruning comprises:calculating a zoom factor according to the first width and the secondwidth; and when the first width is less than the second width stretchingall pixels in a color array of the source image according to the zoomfactor, calculating a part that needs to be pruned from the sourceimage, determining in the color array of the source image a pixel areacorresponding to the part that needs to be pruned, and discarding pixeldata in the pixel area, thereby obtaining the target image correspondingto the source image.
 11. The method according to claim 10, wherein thezooming in and pruning comprises: when the first width is less than thesecond width, zooming in the source image according to a zoom factor k,wherein k=(b+d)/(a+d), a is the first width, b is the second width, andd is the width of the source image; and pruning an edge part from thesource image after zooming in, so as to obtain the target imagecorresponding to the source image.
 12. The method according to claim 1,further comprising: preprocessing original images so as to obtain the Nsource images, the N source images having the same dimensions as targetimages to be displayed on the N target display devices, and whereindimensions of the original images are different from the dimensions ofthe target images.
 13. The method according to claim 12, wherein thepreprocessing comprises: separately zooming in or out, pruning, and/orpadding the original images as to obtain corresponding source images.14. The method according to claim 13, wherein the separately zooming inor out, pruning, and/or padding comprises: if aspect ratios of theoriginal images are the same as aspect ratios of the target images andthe width w of the original images is different from the width d of thetarget images, proportionally zooming in or out, according to a zoomfactor m, the original images so as to obtain the corresponding sourceimages, wherein m=d/w.
 15. The method according to claim 14, wherein thefirst width a is determined according to a=n×A, wherein n=m, and A isthe width of one or more blind spots between the original images. 16.The method according to claim 13, wherein the separately zooming in orout, pruning, and/or padding comprises: if aspect ratios of the originalimages are different from aspect ratios of the target images,proportionally zooming in or out, according to a zoom factor m1, theoriginal images corresponding so as to obtain corresponding intermediateimages, wherein m1=e/h, e is the height of the target images, and h isthe height of the original images; and pruning or padding an edge thathas a total width of |d−(ew/h)| from or into one or two sides of theintermediate images, so as to obtain the corresponding source images,wherein d is the width of the target images and w is the width of theoriginal images.
 17. The method according to claim 16, wherein the firstwidth a is determined according to a=n×A, wherein n=(2Ae+ew−dh)/2Ah, andA is the width of one or more blind spots between the original images.18. An image processing device, comprising a non-transitoryprocessor-readable medium having processor-executable instructionsstored thereon for providing video conferencing, theprocessor-executable instructions, when executed by a processor, causingthe following steps to be performed: determining a first widthcorresponding to width of one or more blind spots between N sourceimages and determining a second width corresponding to width of one ormore gaps between N target display devices, wherein N is an integergreater than 1 and the N target display devices have screens that are ofa same size and are arranged side by side at a same height, and whereinthe first width is different from the second width; and adjusting thesource images according based on the determined first width and secondwidth so as to obtain target images to be displayed on the N targetdisplay devices such that no mismatch exists in the target imagesdisplayed on the N target display devices:, wherein the adjustingcomprises pruning and padding a source image so as to obtain acorresponding target image; wherein the pruning and padding comprises:calculating a part that needs to be pruned from the source image;determining in a color array of the source image a pixel areacorresponding to the part that needs to be pruned; discarding pixel datain the pixel area; calculating a part that needs to be padded;determining in the color array of the source image, a position of apixel area corresponding to the part that needs to be padded; and addingpixel data of a preset color into the pixel area corresponding to theposition of the pixel area thereby obtaining the target imagecorresponding to the source image.
 19. The device according to claim 18,wherein the processor-executable instructions, when executed by aprocessor, further cause the following step to be performed:preprocessing original images so as to obtain the N source images, the Nsource images having the same dimensions as target images to bedisplayed on the N target display devices, and wherein dimensions of theoriginal images are different from the dimensions of the target images.20. The device according to claim 19, wherein the preprocessingcomprises: separately zooming in or out, pruning, and/or padding theoriginal images as to obtain corresponding source images.
 21. The deviceaccording to claim 20, wherein the separately zooming in or out,pruning, and/or padding comprises: if aspect ratios of the originalimages are the same as aspect ratios of the target images and the widthw of the original images is different from the width d of the targetimages, proportionally zooming in or out, according to a zoom factor m,the original images so as to obtain the corresponding source images,wherein m=d/w.
 22. The device according to claim 21, wherein the firstwidth a is determined according to a=n×A, wherein n=m, and A is thewidth of one or more blind spots between the original images.
 23. Thedevice according to claim 20, wherein the separately zooming in or out,pruning, and/or padding comprises: if aspect ratios of the originalimages are different from aspect ratios of the target images,proportionally zooming in or out, according to a zoom factor m1, theoriginal images corresponding so as to obtain corresponding intermediateimages, wherein m1=e/h, e is the height of the target images, and h isthe height of the original images; and pruning or padding an edge thathas a total width of |d−(ew/h)| from or into one or two sides of theintermediate images, so as to obtain the corresponding source images,wherein d is the width of the target images and w is the width of theoriginal images.
 24. The device according to claim 23, wherein the firstwidth a is determined according to a=n×A, wherein n=(2Ae+ew−dh)/2Ah, andA is the width of one or more blind spots between the original images.25. The method according to claim 8, further comprising: preprocessingoriginal images so as to obtain the N source images, the N source imageshaving the same dimensions as target images to be displayed on the Ntarget display devices, and wherein dimensions of the original imagesare different from the dimensions of the target images.
 26. The methodaccording to claim 25, wherein the preprocessing comprises: separatelyzooming in or out, pruning, and/or padding the original images as toobtain corresponding source images.
 27. The method according to claim26, wherein the separately zooming in or out, pruning, and/or paddingcomprises: if aspect ratios of the original images are the same asaspect ratios of the target images and the width w of the originalimages is different from the width d of the target images,proportionally zooming in or out, according to a zoom factor m, theoriginal images so as to obtain the corresponding source images, whereinm=d/w.
 28. The method according to claim 10, further comprising:preprocessing original images so as to obtain the N source images, the Nsource images having the same dimensions as target images to bedisplayed on the N target display devices, and wherein dimensions of theoriginal images are different from the dimensions of the target images.29. The method according to claim 28, wherein the preprocessingcomprises: separately zooming in or out, pruning, and/or padding theoriginal images as to obtain corresponding source images.
 30. The methodaccording to claim 29, wherein the separately zooming in or out,pruning, and/or padding comprises: if aspect ratios of the originalimages are the same as aspect ratios of the target images and the widthw of the original images is different from the width d of the targetimages, proportionally zooming in or out, according to a zoom factor m,the original images so as to obtain the corresponding source images,wherein m=d/w.